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10 | Director of the Office of Training at CIA, which Eadeed | 
| f 
i $ 
on dees~conduct;training courses for intelligence analysts, ané | 
12 | td feel that the training conducted in those courses is 
| (wee 4 Af 
13 | applicable for indications anetyeee asi for other types of 
is | intelligence analysis. And If think the statement that is | 
: { 
| ! 
1S underlined on page 6 of your report points this out very | 
16 | nicely. 
H . 
17 | We offer three different intelligence analysis courses/ 
i 
Lye 
} 
1g } and se a aay rar ois for analysts, although there are dupe 
| a tae | 
19 | trainingveourssd fom aanysts as well. In these courses we ! 
h 
20 | exy to cover the very things that I have heard you ‘talking 
2) | to the Defense representatives here today. For example, we 
| | 
: a9 i cover such things as methodologies that are available to and | 
} i 
“4 help or analysis, pe re ee have one very interesting |! 


exercise Wirteh=:s. ako run on a computer, a PDP-1145, which 
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! people to change the values of their ee. based upon 


Kcrentt’ ab greeter Ma Prdatey Pee, 


receipt of successive data. ‘This is Something.that analysts ~- 


and I am an old intelligence analyst ata ~know~——thatyou~tend 


. thay 
to hang on to the hypothesis that yor start out with, and 


hag Wor 4 ha E 
as youe get new data, you-dor't change yous hypothesis. Well, 


thig is an attempt to teach people how to do that. 

Another area that we teach them-in is what is called 
subjective probability assessment, to sharpen up their judg- 
ments. If I could explain it like this, they are aekea ke 


mo . here 
make judgments abert -- and this is in sceveieds factual 
7a | 


statements or sevens / euaeo place a probability upon the 


ee 


Likelihood of these things, and then having done so, to give 


Ae Es EN RAE “OR OO I 


an indication of the confidence that they have in their own 
judgiient. amd Sfou can tesan’ peep through this technique 
to improve their judgments and 'treetfeet-upen the confidence 


WA oo) Mixactal 
thet they have in-it; and to do a little bit better job of 


ee = eee RES a NN Se 


doing that and in communicating it to the people that inerenent 
intelligence product, thet~etens from this 

Finally, in this area, we have worked on the question of 
rn se en er eee rer ere ees is. 
pretty easily identified, Sr eer parts of 
organizations have different biases, but above all, the really 
difficult one is the bias that resides in each individual. It 


is harder to identify and it is harder to do something about 
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expexrience#{ perhaps cognitive bias, you might say. 


And finally, one of the kinds of things that we do in 


ee 


these programs ~~ we have three different courses here -~- is 
Autly vt deni bbs af 
“in the area of creativity, of helping “them to tirink-up new 
Zan 


hypotheses, te-address—that. In that connection we have got 


of what makes people thinkYfcreatively, and ftlhey have been very 
cooperative with us and have given us the benefit of a lot 
of their research and of training programs that they have 
conducted. | . —_ _ 
‘We originally sent people down there for training, and 
subsequently, one of their principal training programs was 
transferred in-house and is conducted now at CIA six times a 
year with the assistance of these people. 

We have three different eoaeees for different levels of 


people. These are all intelligence analysis course’ and they 


vary in length from two weeks to four weeks, and the numbers 


yee cee mares cersereen seeemneneeererneaniemanaereteensit gain cere aeeas area Sancta ee a 


OF SPUOI MW Fane ( ta Ohad Atte 
yary~eround’15 to 20. We have a total capacity of trainingrfor 


_ 


about 300 people a year. We do not train ‘pecifically, as the 


bur- 


teeete statement said, in indications and warning, and we do 


people over to Capta i course at the Defense Intelligence 


n 
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Mr. Rose. Do all analysts at the CIA benefit from these 
3/ 
ca 


a Not all analysts have taken the courses, 


particularly people who have been around for a while. Some of 


these are fairly new programs that have not been under way for 
Yat, af 4 Shae 
along time. I guess we would Like to thinkjwe — are doing the. 


Mele ye” 
ate that if they take them, they would certainly benefit 


a them. 


I think we start from the point of view that we expect 


people to be able to analyze and to think when they are hired. | 


That is one of the things that we don't ékpect to have to teach 
them after they get there. 

Mr. Rose. How do you tole them to Dee See that ey Gon 
do that when you hire them? 

($$ Well, there's a variety of ways. One is we 
do give a variety of psychological tests to them in the first 


place to establish their IQ. I am not an expert on this, sir, 


| 
} 
H 
‘ 
and I cantt go into it in great depth. We do a lot of aces 
t 
viewing of them at that time aes people who are experienced in 
Wh, suite bentn the) 
thie-field. We would not rely on test batteries by themselves.; 
aE. | 
And we obviously look at their previous. experience, RenCEE TORE 
WES 
Vecntsek with the people hae 


| 


academic transcripts, and above all 


they have worked for or in their communities, r think re of 
~~ Fhest ou? bat ated Ger a gil 
interest to us; contact “bas sicadkly ‘Look seg at their background 
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5X1A : ‘ 
——= No, sir, they are not. We ran an experiment 


in this, not with the NID because it didn't exist at the 
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3 Mr. Rose. What are the. titles of those courses? Do you 
| : 
4 | have them? 
25X1A 
5 | oo a Yes, indeed. ~Seminar on Intelligence 
6 | Analysis, which runs four times each year for three weeksy 
7 | Introduction to Analysis for Career Trainees -- those are . | 
§ | entry level people -- which is-runm--—ftt is two weeks and | 
| Wh Thal w 1 
9 | it—ts* run four times a year. Weehave-—something called the 
{ : ' - : 
19 | Intelligence Process Course, which is conducted twice a year, | 
| 
nN four weeks in length. | 
12 | Mr. Rose. Does the Intelligence Process Course, which | 
13 | teaches analysts to make finished intelligence meaningful 
_ ; ; 
id to executive-level policy makers, does it deal specifically 
| 
a5x1A # with warning? | 
! | 
{ 
16 | Yes, but only inva very short fashion. Therej 
i , | 
7 | is not:much coverage on warning in it. { 
| | 
13 | Mr. Rose. The National Intelligence Daily, which serves a 
H 
19 | high-level audience, could be considered to have a warning 
t ; . | [ 
20 ‘function. Are’ the articles in the NID evaluated for their 
i ; és : 4 . ’ 
i 
3] | predictive accuracy? 
| { 
22 | 25x1) I had better come up to the table. | 
| 
3 | Mr. Rose. Come on. 
| | 
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: | 
| time, but with its predecessor, the National Intelligence 


oy 


Bulletin, and we found that it was not a fruitful exercise, | 


v 


da 


3 that you ran into too many the-sun-will-rise-tomorrow predic- 
| tions. If the analysts knew we were grading them on this, we 
t 


would::run into a great many more of them, and there were any 


pilots to compensate for illusions such as things appearing 


to be closer on clear days thanion hazy days. 


| number of pitfalls in it that made the statistics that we 
; came up with quite meaningless. 
| 
7 reduce or compensate for the effects of perceptual bias in 
7 human mental processes, as you mentioned. For eienpies 
ny repetitive training with constant feedback allows airline 
12 | 

| 


Mr. Rose. Some procedures have been developed to 
‘ 1 
| 

| 

| 
“Does the CIA provide any training to its analysts which : 
! 

i 


25X1A 


\ 
is 
| e e s ’ 
1§ } Will actually help them overcome cognitive biases? 
i 
j ’ 
a. I referred a moment ago, sir, to two exercises 


/ tele Leak | 
which are Pee ee live, at,Say, a computer terminal keyboard, 


this business of adjusting one's hypothesis based upon the 


which provides instant feedback. One of those exercises is 
receipt of new data. It is based on Bayes! work, or Bayesian - 
algebra. The other is a subjective probability test where one 


is able to get instant feedback. 


ts 


are very limited and they may be somewhat primitive, but I do 
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"3 i I think these are in that general ballpark. I think they 
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Sen en a 
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work is concerned. 
Mr. Rose. All right, thank you very much. 
(Whereupon, at 3:14 binloek p.m., the Subcommittee 


proceeded to the consideration of other matters.) 
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